Voice Conversion Using A Two-Factor Gaussian Process Latent Variable Model

This paper presents a novel strategy for voice conversion by solving style and content separation task using a two-factor Gaussian Process Latent Variable Model (GP-LVM). A generative model for speech is developed by interaction of style and content, which represent the voice individual characteristics and semantic information respectively. The interaction is captured by a GP-LVM with two latent variables, as well as a GP mapping to observation. Then, for a given collection of labelled observations, the separation task is accomplished by fitting the model with Maximum Likelihood method. Finally, voice conversion is implemented by style alternation, and the desired speech is reconstructed with the decomposed target speaker style and the source speech content using the learned model as a prior. Both objective and subjective test results show the advantage of the proposed method compared to the traditional GMM-based mapping system with limited size of training data. Furthermore, experimental results indicate that the GP-LVM with nonlinear kernel functions behaves better than that with linear ones for voice conversion due to its ability of better capturing the interaction between style and content, and rich varieties of the two factors in a training set also help to improve the conversion performance. Streszczenie. W artykule opisano nową strategię konwersji głosu, poprzez rozdzielenie rodzaju i treści, przy wykorzystaniu dwu-wskaźnikowej metody GPLVM (ang. Gaussian Process Latent Variable Model). Wykonane badania wskazują na lepsze działanie proponowanego algorytmu w porównaniu z tradycyjnie stosowanym systemem mapowania typu GMM przy ograniczonej ilości danych do testowania. Wykazano, że GPLVM ma lepsze właściwości w konwersji głosu z nieliniową niż liniową funkcją jądra. (Dwuwskaźnikowa metoda GPLVM w procesie konwersji głosu).